Single Producer, Single Consumer Lockless FIFO/LIFO Queue

ABSTRACT

A query inserter receives data elements having individual priority types for placement in a queue, and utilizes the priority types of the received data elements to determine placement in the queue relative to an initial location established when a first data element is placed in an empty queue in order to manage the queue with a combination of first-in first-out and last-in first-out queue functionality.

BACKGROUND

A thread may add requests to a queue, and another thread may removerequests from the queue. Manipulating a request may require multipleinstructions. The queue is a shared resource and a race condition mayoccur if, for example, a thread attempts to read from the queue whileanother thread attempts to add a request to the queue. For example, ifone thread attempts to read from the queue while another is in themiddle of manipulating the queue, the reading thread may find the queuein an inconsistent state. This may result in corruption of the queue.

A common mechanism for ensuring that only one thread manipulates thequeue at a time is to provide a locking mechanism for the queue. Forexample, if a thread attempts to add a request to the queue, the threadfirst obtains a lock and then safely adds the request to the queue.After adding the request, the thread releases the lock to allow anotherthread to manipulate the queue. Locking, however, is relativelyexpensive and can limit scalability. The more locks that are implementedin a system, the more time is spent acquiring and releasing those locks,which is a relatively expensive operation.

Further, an example of a widely used queue is a first-in, first-out(FIFO) queue. In a FIFO queue, data elements inserted before other dataelements are removed before the other data elements. Accordingly, a highpriority request in a FIFO queue must wait until other requests in thequeue have been removed before the high priority request is serviced.This may lead to undesirable results.

BRIEF SUMMARY

System, method, and computer program product embodiments for managing alockless queue are provided. In an embodiment, a single producer threadand a single consumer thread manipulate the lockless queue. According toan example method, data elements having individual priority types arereceived for placement in a queue. The priority types of the receiveddata elements are utilized to determine placement in the queue relativeto an initial location established when a first data element is placedin an empty queue in order to manage the queue with a combination offirst-in first-out and last-in first-out queue functionality.

Further features and advantages, as well as the structure and operationof various embodiments thereof, are described in detail below withreference to the accompanying drawings. It is noted that the embodimentsare not limited to the specific embodiments described herein. Suchembodiments are presented herein for illustrative purposes only.Additional embodiments will be apparent to persons skilled in therelevant art(s) based on the teachings contained herein.

BRIEF DESCRIPTION OF THE DRAWINGS/FIGURES

FIG. 1 illustrates an example First-in, First-out queue.

FIG. 2 illustrates a FIFO/Last-in, First-out (LIFO) queue arrangement,according to an example embodiment.

FIG. 3 illustrates a FIFO/LIFO queue after two data elements of thefirst priority type have been inserted in a queue, according to anexample embodiment.

FIG. 4 illustrates a FIFO/LIFO queue after a data element of the secondpriority type has been inserted in a queue, according to an exampleembodiment.

FIG. 5 illustrates a FIFO/LIFO queue after a linked list in a queue hasbeen closed, according to an example embodiment.

FIG. 6 illustrates a FIFO/LIFO queue after a linked list has beenremoved from the queue and a data element of the second priority typehas been inserted in the queue, according to an example embodiment.

FIG. 7 illustrates a system for managing a lockless FIFO/LIFO queue,according to example embodiments.

FIG. 8 illustrates a flowchart of a method for managing a locklessFIFO/LIFO queue, according to an example embodiment.

FIG. 9 illustrates a flowchart of a method for removing data elementsfrom a lockless FIFO/LIFO queue, according to an example embodiment.

FIG. 10 illustrates an example computer system, according to an exampleembodiment.

Features and advantages of embodiments will become more apparent fromthe detailed description set forth below when taken in conjunction withthe drawings. In the drawings, like reference numbers generally indicateidentical, functionally similar, and/or structurally similar elements.Generally, the drawing in which an element first appears is indicated bythe leftmost digit(s) in the corresponding reference number.

DETAILED DESCRIPTION I. Overview II. FIFO Queue III. FIFO/LIFO Queue

A. Priority Types

B. Insertion of Data Elements

-   -   1. Insertion of Data Elements of the First Priority Type    -   2. Insertion of Data Elements of the Second Priority Type

C. Removal of Data Elements

IV. Example System Embodiment V. Example Method Embodiment VI. ExampleComputer Implementation Embodiment

I. Overview

The following detailed description refers to the accompanying drawingsthat illustrate exemplary embodiments. Other embodiments are possible,and modifications can be made to the embodiments within the spirit andscope of the embodiments. Therefore, the detailed description is notmeant to limit the embodiments. Rather, the scope of the embodiments isdefined by the appended claims.

A queue is a collection in which data elements are stored and retrievedin a specific order. In an example, a producer thread may produce one ormore data elements and place the one or more data elements in a datastructure, such as a queue. A consumer thread may remove a data elementfrom the queue and process the data element.

Described herein are embodiments for managing a lockless FIFO/LIFOqueue. In an example, managing a lockless FIFO/LIFO queue may includereceiving data elements having individual priority types for placementin a queue, and utilizing the priority types of the received dataelements to determine placement in the queue relative to an initiallocation established when a first data element is placed in an emptyqueue in order to manage the queue with a combination of first-infirst-out and last-in first-out queue functionality. In an example, afirst-in first-out data element may have a first priority type and alast-in first-out data element may have a second priority type.

Further, this disclosure describes a data element of a second prioritytype having a higher priority than a data element of a first prioritytype. It should also be recognized that in some embodiments, a dataelement of a first priority type may have a higher priority than a dataelement of a second priority type.

II. FIFO Queue

FIG. 1 illustrates an example First-in, First-out (FIFO) queue 100. In aFIFO queue, a data element that is inserted before other data elementsin the FIFO queue will be removed before the other data elements. Inthis disclosure, a data element that is said to be “inserted before”another data element in the queue is removed from the queue before theother data element. Similarly, a data element that is said to be“inserted after” another data element in the queue is removed from thequeue after the other data element.

In FIFO queue 100, data elements are inserted at a first end (e.g.,tail) 110 of FIFO queue 100 and removed at a second end (e.g., head) 115of FIFO queue 100.

Queue 100 includes data elements 105 a-105 d. In an example, a producerthread produces data elements 105 a-105 d, respectively, and insertsthem at tail end 110 of FIFO queue 100, respectively. The producerthread may then produce data element 105 e, and insert it at tail end110 of FIFO queue 100 after data element 105 d.

A consumer thread may remove data elements from head end 115 of FIFOqueue 100. In an example, the consumer thread removes data element 105 abefore removing other data elements. After removing data element 105 a,the consumer thread may then remove data elements 105 b-105 d,respectively.

The inventor recognized that a FIFO queue may contribute to unnecessarydelays in an urgent data element being removed from the queue andprocessed. For example, the producer thread may produce a high prioritydata element 105 e, and thus it may be undesirable for high prioritydata element 105 e to wait until all other data elements (e.g., dataelements 105 a-105 d) in FIFO queue 100 have been removed.

One solution provided by the inventor in this disclosure includesallowing data elements to have different priority types such that dataelements having a higher priority type are added before (closer to thehead of the queue) data elements having a lower priority type in thequeue. Thus, the removal of data elements may be prioritized based on apriority type of each data element. Further, a lockless queue isprovided so that insertions of data elements in the queue are notstalled waiting for other data elements to be read from the queue.

In an embodiment, a single producer and a single consumer operate on alockless FIFO/LIFO queue. More details on the queue arrangement arebelow.

III. FIFO/LIFO Queue

A. Priority Types

According to an embodiment, data elements having individual prioritytypes are inserted in the queue. The priority type of a data element isconsidered when it is inserted in the queue. In an example, the prioritytypes of the data elements are utilized to determine placement in thequeue relative to an initial location established when a first dataelement is placed in an empty queue.

In an embodiment, a data element inserted in a queue may be of a firstpriority type or a second priority type. A priority type may indicate anurgency of the data element. For example, a data element of the secondpriority type may have a greater urgency for servicing than a dataelement of the first priority type. Accordingly, the data element of thesecond priority type may be serviced before a data element of the firstpriority type.

FIG. 2 illustrates a FIFO/Last-in, First-out (LIFO) queue arrangement,according to an example embodiment. In a typical LIFO queue, a dataelement is inserted and removed at one end of the queue, such that thelast element inserted is the first element removed.

FIG. 2 illustrates a lockless FIFO/LIFO queue 201 that is empty and doesnot include any data elements. Queue 201 may include data elements ofdifferent priority types. In an embodiment, queue 201 is a circularbuffer including one or more linked lists.

A producer thread may produce one or more data elements and insert thedata elements in queue 201. A consumer thread may remove data elementsfrom queue 201 and process the one or more data elements.

FIG. 2 also shows a primary array 205, a tail array 210, and indicesput-id 215 and get-id 220. Primary array 205 includes primary arrayelements 205 a-205 n and may store up to N references to data elementsof a first priority type that are inserted in the queue. Primary arrayelement 205 a is at position zero in primary array 205, primary arrayelement 205 b is at position one in primary array 205, and primary arrayelement 205 n is at position n in primary array 205. Index put-id 215references the next available position in primary array 205 that may beused to insert a data element in queue 201.

Queue 201 may include a linked list of data elements (not shown in FIG.2). Tail array 210 includes tail array elements 210 a-210 n and maystore a reference to an end of the linked list in queue 201. Tailelement 210 a is at position zero in tail array 210, tail array element210 b is at position one in tail array 210, and tail array element 210 nis at position n in tail array 210. The linked list in queue 201 mayinclude one or more data elements that include references. Index get-id220 references a position in primary array 205 that may be used toremove a data element from queue 201.

FIG. 2 also shows a stop node 225 and a list buffer 230. In anembodiment, data elements may be added or removed from queue 201 withoutrequiring locks. Stop node 225 and list buffer 230 may assist inproviding this advantage. Accordingly, it may be unnecessary for aproducer thread to wait until all data elements to be removed have beenremoved from queue 201 before inserting a data element in queue 201.

Stop node 225 may close the linked list in queue 201, preventingadditional data elements from being added to the linked list in queue201. In an embodiment, preventing additional data elements from beingadded to the linked list in queue 201 includes identifying a mostrecently inserted data element in the linked list in queue 201 andupdating the reference included in the most recently inserted dataelement to reference stop node 225.

When a data element in the linked list in queue 201 references stop node225, this indicates to the producer thread that the linked list in queue201 is closed. The producer thread may then insert data elements atanother position in queue 201. In this way, queue 201 is accessible evenwhen the consumer thread reads data elements from queue 201. Subject tomemory constraints, queue 201 does not block when data elements areinserted into the linked list in queue 201. Accordingly, data elementshaving a high urgency (e.g., data elements of the second priority type)will always be enqueued, and it is unnecessary for the producer threadto wait until the data elements have been read from queue 201 beforeinserting data elements in queue 201.

List buffer 230 may reference a head of the linked list of data elementsin queue 201, and the linked list may be delinked from primary array205. As such, list buffer 230 may reference the linked list that is nolonger in queue 201. The data elements in the linked list may betraversed and processed without conflicting with data elements beinginserted in queue 201.

The FIFO/LIFO queue arrangement of FIG. 2 may be lockless and use anatomic compare-and-swap instruction. A compare-and-swap instructionensures that the threads will have a consistent view of any location inqueue 201 and do not change the shared resource at the same time. Forexample, contention may occur if at approximately the same time, theproducer thread inserts a data element having a second priority type inqueue 201 by trying to append the data element at the tail of the linkedlist and the consumer thread updates the most recently inserted dataelement in the linked list in queue 201 to reference stop node 225. Thecompare-and-swap instruction may be used to determine whether the mostrecently inserted data element in the linked list references stop node225 or a NULL value. If the most recently inserted data element in thelinked list references stop node 225, then the linked list is closed andthe producer thread does not insert a data element into the linked list.Rather, the producer thread inserts a data element at the next availableposition in queue 201. For example, the producer thread may updateanother linked list at the next available position in queue 201 andinsert the data element into this linked list.

Further, read and write of integral values of less than or equal to thebus width may be atomic operations.

Data to be inserted in queue 201 may be stored in a data structure. Adata element in the linked list may, for example, include a reference toanother data element in the linked list, a value such as NULL, or stopnode 225. A reference may be, for example, a pointer to another dataelement. The data to be inserted in queue 201 may be stored in a nodehaving a reference to another node (e.g., the next node in queue 201) orto a value such as NULL. In an example, a Node data element includes a“next” pointer that points to the next data element in queue 201. Thenext data element in queue 201 may be the next data element to beremoved from queue 201 after the data element that references the nextdata element is removed.

B. Insertion of Data Elements

Data elements may be inserted in queue 201 based on a priority type ofthe data element. According to an embodiment, inserting the data elementin queue 201 may include determining whether queue 201 is empty. Whenqueue 201 is determined to be empty, a received data element is insertedat an initial location in queue 201. As such, when queue 201 is empty,inserting a data element of a first priority type may be functionallythe same as inserting a data element of the second priority type inqueue 201.

When queue 201 is determined to be not empty, it is determined whetherthe received data element is of a first priority type or a secondpriority type. Utilizing the priority types of the received dataelements to determine placement in the queue relative to an initiallocation may include when the received data element is determined to beof the first priority type, the received data element is inserted inqueue 201 after each existing data element of the first priority type inqueue 201 (i.e., closer to the tail end of queue 201). Utilizing thepriority types of the received data elements to determine placement inthe queue relative to an initial location may also include when thereceived data element is determined to be of the second priority type,the received data element is inserted in queue 201 before each existingdata element of the first priority type after the initial location inqueue 201 (i.e., closer to the head end of queue 201).

1. Insertion of Data Elements of the First Priority Type

Related to FIG. 2, FIG. 3 illustrates a FIFO/LIFO queue 201 after twodata elements of the first priority type 345, 350 have been inserted inqueue 201 from FIG. 2, according to an example embodiment. Referring toFIG. 2, prior to insertion of elements 345 and 350, queue 201 is emptyand index put-id 215 is equal to zero, indicating that position zero inprimary array 205 is the next available position in primary array 205that may be used to insert a data element in queue 201.

In the example of FIG. 3, a producer thread produces data elements ofthe first priority type 345 and 350. The producer thread inserts dataelement 345 in queue 201 at an initial location, and both primary array205 and tail array 210 are updated to point to data element 345. Inparticular, primary array element 205 a at position zero in primaryarray 205 is updated to reference data element 345, and tail arrayelement 210 a at position zero is updated to reference data element 345,the most recently inserted data element in the linked list in queue 201.Accordingly, a first linked list including one data element is generatedthat includes data element 345. Also, index put-id 215 is incremented toone, indicating that position one in primary array 205 is the nextavailable position in primary array 205 that may be used to insert adata element in queue 201. In an embodiment, index “put-id” 215 isincremented only after primary array 205 and tail array 210 have beenupdated to point to the data element 345. In this way, the consumerthread will not attempt to read the data element before producer threadis done inserting the data element in queue 201.

The producer thread then produces data element 350. The producer threaddetermines whether data element 350 is of a first priority type or asecond priority type. For purposes of illustration, assume data element350 is of the first priority type, indicated by a “1” in data element350. The producer thread then inserts data element 350 in queue 201after each existing data element of the first priority type in queue201. In particular, the producer thread may insert data element 350 inqueue 201 after data element 345, closer to the tail end of queue 201.Both primary array 205 and tail array 210 are updated to point to dataelement 350. In particular, primary array element 205 b at position onein primary array 205 is updated to reference data element 350, and tailarray element 210 b at position one in tail array 210 is updated toreference data element 350 at the tail end of a linked list, differentfrom the linked list that includes data element 345, in queue 201.Accordingly, a second linked list including one data element isgenerated that includes data element 350.

Index put-id 215 is incremented to two, indicating that position two inprimary array 205 is the next available position in primary array 205that may be used to insert a data element of the first priority type inqueue 201.

In an embodiment, the example code shown below (or functionally similarcode) may be used to insert a data element of the first priority type inqueue 201 after each existing data element of the first priority type inqueue 201

bool insert_firstprioritytype(Node * node) { if (putid + 1 % SIZE ==getid) // queue is full return false; array[putid] = node; tails[putid]= node; // insert_firstprioritytype ( ) always places a //node into anempty slot, //point the tail for this slot to the new node putid =putid + 1% SIZE; // increment put index and roll over, if needed returntrue; }

After executing the above code (or functionally similar code), a dataelement of the first priority type is inserted into the next availableposition in queue 201, and index put-id 215 is incremented modulo N.Further, primary array 205 and tail array 210 are updated to point to anew data element of the first priority type. If queue 201 is full, afalse value is returned.

2. Insertion of Data Elements of the Second Priority Type

FIG. 4 illustrates the example FIFO/LIFO queue 201 after a data elementof the second priority type 455 has been inserted in queue 201 in FIG.3, according to an example embodiment.

The producer thread produces data element of the second priority type455. To insert a data element in queue 201, it is determined whetherqueue 201 is empty. Referring back to the example of FIG. 3, queue 201is not empty and includes data elements 345 and 350. The producer threadthen determines whether new data element 455 is of a first priority typeor a second priority type. For illustrative purposes, assume dataelement 455 is of the second priority type, indicated by a “2” in dataelement 455.

The producer thread then inserts data element 455 in queue 201 after theinitial location in queue 201. The initial location in queue 201 is alocation of an oldest data element relative to the existing dataelements in queue 201. In an example, in queue 201, data element 345 isat an initial location in queue 201. Accordingly, the producer threadinserts data element 455 in queue 201 before each existing data elementof the first priority type after data element 345.

In the example of FIG. 4, inserting data element 455 in queue 201 beforeeach existing data element of the first priority type after the initiallocation in queue 201 involves inserting data element 455 into the firstlinked list including data element 345. In particular, the producerthread may insert data element 455 at the tail end of the first linkedlist.

In an example, a most recently inserted data element in the first linkedlist may be identified. Data element 345 is the most recently inserteddata element in the first linked list. The first linked list includes anoldest data element in the queue. The oldest data element in queue 201may be identified by determining the value of index get-id 220. Forexample, in FIG. 3, index get-id 220 is zero. Accordingly, position zeroin tail array 210, tail array element 210 a, references data element345, the most recently inserted data element in the first linked list.Data element 345 is also the oldest data element in queue 201. Dataelement 455 is inserted into the first linked list by updating thereference included in data element 345 to reference the newly inserteddata element 445 of the second priority type using, for example, thecompare and swap operation. In an example, the compare and swapoperation checks whether the reference included in data element 345points to NULL. If so, the first linked list is not closed and anadditional data element (e.g., data element 455) can be appended to thefirst linked list. Tail array element 210 a is then updated to point todata element 455, which is now the new tail of the first linked list.

Thereafter, queue 201 includes data elements 345, 350, and 455, primaryarray 205 a references data element 345, and tail array 210 a referencesdata element 455, the most recently inserted data element in the firstlinked list.

The producer thread may insert another data element in the first linkedlist by inserting the new data element at the new tail end of the firstlinked list. For example, in FIG. 4, another new data element of thesecond priority type may be inserted into the first linked list afterdata element 455 by updating the reference included in data element 455to reference the newly inserted data element of the second prioritytype. Tail array 210 a may then be updated to reference the newlyinserted data element of the second priority type.

In an embodiment, the code shown below (or functionally similar code)may be used to insert a data element of the second priority type inqueue 201 before each existing data element of the first priority type.

bool insert_secondprioritytype (Node * node) { int getlocal = getid;  //store the get index locally if (putid == getid) { // the queue is empty,only insert_xxx operations can array[putid] = node; // change thisstate, so there is no contention with dequeue tails[putid] = node; putid= putid + 1% SIZE; return true; } else { // the queue is not empty, weare going to Node * temp = tails[getlocal]; // put this on the tail Node * next = temp−>next; if (next == &stopNode)  // dequeue( ) hasmarked this list as  closed, try again return false; // this time in thenext slot. if CAS(&temp−>next, NULL, &stopNode) //CAS is CompareAndSwap{ tails[getlocal] = value; // we have managed to insert the node at theend of return true; // the list, swing the tail pointer to us } } returnfalse; }

After executing the above code (or functionally similar code), a dataelement of the second priority type is inserted into the linked list inqueue 201. If queue 201 is empty, inserting a data element of the secondpriority type is functionally the same as inserting a data element ofthe first priority type in queue 201. If queue 201 is not empty, indexget-id 220 is used to determine the location of the oldest data elementof the first priority type relative to the existing data elements of thefirst priority type in queue 201. The newly inserted data element of thesecond data type is linked to the oldest data element of the firstpriority type, creating the linked list. To create the linked list, thecurrent terminator is checked for NULL (e.g., by using atomiccompare-and-swap), and the new data element of the second priority typeis inserted at the end of the linked list. If the current terminator isstop node 225, the linked list has been closed, and a false value isreturned. The next call will attempt to insert the data element of thesecond priority type at the next available location in the queue.

C. Removal of Data Elements

Data elements may be removed from the queue based on a priority type ofthe data element. Removing one or more data elements from the queue mayinclude removing a linked list from the queue. The linked list mayinclude an oldest data element in the queue.

In an embodiment, removing one or more data elements includes preventingadditional data elements from being added to the linked list includingone or more data elements. The linked list may include a data element ofthe first priority type and zero or more data elements of the secondpriority type. In an example, an oldest data element relative to theexisting data elements in queue 201 is identified. The oldest dataelement is at a head of the linked list. The linked list is removed fromqueue 201. The removed linked list may be traversed starting at the headof the removed linked list. One or more data elements in the removedlinked list is processed based on an order in which the removed linkedlist is traversed.

FIG. 5 illustrates FIFO/LIFO queue 201 after the linked list in queue201 in FIG. 4 has been closed, according to an example embodiment.Closing the linked list prevents additional data elements from beingadded to the linked list. Referring back to FIG. 4, queue 201 includesdata elements 345, 350, and 455. Data elements 345 and 350 are of thefirst priority type, and data element 455 is of the second prioritytype.

In an embodiment, preventing additional data elements from being addedto the linked list includes identifying a most recently inserted dataelement in the linked list, and updating a reference included in themost recently inserted data element to reference stop node 225. The mostrecently inserted data element in the linked list is identified bydetermining the value of index get-id 220. For example, in FIG. 4, indexget-id 220 is zero. Accordingly, position zero in tail array 210, tailarray element 205 a, references data element 455, the most recentlyinserted data element in the linked list in queue 201.

In the example of FIG. 5, data element 455 is the most recently inserteddata element in the linked list. The reference included in data element455 is updated to reference stop node 225 using the compare and swapoperation. In this way, additional data elements are prevented frombeing added to the linked list.

FIG. 6 illustrates FIFO/LIFO queue 201 after linked list 602 has beenremoved from queue 201 and a data element of the second priority type660 has been inserted in queue 201, according to an example embodiment.

In an embodiment, a list buffer 230 including a reference is identified,and the reference included in list buffer 230 is updated to referencethe data element at the head of linked list 602. For example, listbuffer 230 may reference the data element at the initial location inqueue 201. In an example, in FIG. 6, list buffer 230 is identified andupdated to reference data element 345 at the head of linked list 602.Data element 345 at the head of linked list 602 is the data element atthe initial location in queue 201 and is the oldest data element inqueue 201. Linked list 602 includes data elements 345 and 455, and isremoved from queue 201. Further, index get-id 220 is incremented to one,indicating a position in primary array 205 that may be used to removethe next data element from queue 201.

In an embodiment, after additional data elements are prevented frombeing added to linked list 602, another data element may be inserted inqueue 201. Inserting another data element in queue 201 includesdetermining whether queue 201 is empty. When queue 201 is determined tobe empty, the data element may be inserted at a second location, afterthe initial location, in queue 201. When queue 201 is determined to benot empty, it is determined whether the data element is of a firstpriority type or a second priority type. When the data element isdetermined to be of the first priority type, the data element isinserted in queue 201 after each existing data element of the firstpriority type in queue 201.

When the data element is determined to be of the second priority type,the data element is inserted into a second linked list, different fromthe first linked list, in queue 201. If linked list 602 has been removedfrom queue 201, a head of the second linked list is the oldest dataelement relative to the existing data elements in queue 201.

In the example of FIG. 6, data element 660 is of a second priority typeand is inserted in the linked list in queue 201 by updating a referenceincluded in data element 350 to reference data element 660. Further,tail array 210 b at position one of tail array 210 is updated toreference data element 660, the most recently inserted data element inthe linked list. Further, index put-id 215 is two, indicating thatposition two in primary array 205 is the next available position thatmay be used to insert a data element in queue 201.

In an embodiment, the code shown below (or functionally similar code)may be used to remove a data element of a first priority type or asecond priority type from queue

bool dequeue(Node ** node) { if (NULL != list_buffer) // there are nodesstill in the list we buffered, return dequeue_from_buffer(node); //return from the linked list if ( putid == getid ) // queue is emptyreturn false; // there is an element or a list at index ‘getid’. Movethe list_buffer to point to it // we will use the list buffer toretrieve the elements int getlocal = getid; // save current getindexlocally list_buffer = array[getid]; // list buffer points to the elementto retrieve getid = getid + 1% SIZE; getid = getlocal; // incrementgetid, insert_firstprioritytype( ) will use be able to  // use thisslot. That is okay since we have already cached the  //elements inlist_buffer  //insert_secondprioritytype( ) will use this slot if thequeue becomes  // empty otherwise it will try to use the tail pointer //loop until we are able to close the linked list while (true) { Node *temp = tails[getlocal]; Node * next = temp−>next; if ( CAS(&temp−>next,NULL, &stopNode) ) break; } // the linked list in now cached in listbuffer. Return elements from list buffer return get_from_buffer(value);} // This is executed only in dequeuing thread with no contention boolget_from_buffer(Node ** value) { // we have returned everything from thelist buffer, reset it if (list_buffer == &stopNode) {  list_buffer =NULL; return false; } * value = list_buffer; list_buffer =list_buffer−>next; return true;

After executing the above code (or functionally similar code), a nextdata element is retrieved, if available; otherwise a false value isreturned. If queue 201 has no data elements of the second priority type,the data element at index get-id 220 is retrieved and index get-id 220is incremented modulo N. If queue 201 has one or more data elements ofthe second priority type, they are retrieved from the linked list. In anembodiment, the linked list is closed from further insertions when oneor more data elements is removed from queue 201. This may involve havingthe last data element in the linked list point to stop-node 225 usingthe atomic compare-and-swap operator and incrementing “get-id” 220modulo N. Subsequent insertions of data elements of the second prioritytype will insert the new data element of the second priority type at thenext location. The linked list is then detached from primary array 205and tail array 210 and cached using list buffer 230. Element retrievaltakes place from list buffer 230, with no contention with primary array205 and tail array 210.

IV. Example System Embodiment

FIG. 7 illustrates a system 700 for managing a lockless FIFO/LIFO queue,according to example embodiments.

System 700 includes a client device 705. Client device 705 is acomputing device. Examples of computing devices include, but are notlimited to, a central processing unit, an application-specificintegrated circuit, a computer, workstation, distributed computingsystem, computer cluster, embedded system, stand-alone electronicdevice, networked device, mobile device (e.g. mobile phone, smart phone,personal digital assistant (PDA), navigation device, tablet or mobilecomputing device), rack server, set-top box, or other type of computersystem having at least one processor and memory. A computing processperformed by a clustered computing environment or server farm may becarried out across multiple processors located at the same or differentlocations. Such a computing device may include software, firmware,hardware, or a combination thereof. Software may include one or moreapplications and an operating system. Hardware can include, but is notlimited to, a processor, memory and user interface display.

Client device 705 includes a queue manager 710. Queue manager 710determines whether a queue is empty. When the queue is determined to beempty, queue manager 710 inserts a received data element at an initiallocation in the queue. When the queue is determined to be not empty,queue manager 710 determines whether the received data element is of afirst priority type or a second priority type.

Queue manager 710 also includes a queue inserter 720 and a queue remover725. Queue inserter 720 inserts data elements in a queue based on apriority type of the data element. When the data element is determinedto be of the first priority type, queue inserter 720 inserts the dataelement in the queue after each existing data element of the firstpriority type in the queue. When the data element is determined to be ofthe second priority type, queue inserter 720 inserts the data element inthe queue before each existing data element of the first priority typeafter the initial location in the queue.

Queue remover 725 removes one or more data elements from the queue.

V. Example Method Embodiment

Operations for the above-described embodiments may be further describedwith reference to one or more logic flows. It may be appreciated thatthe representative logic flows do not necessarily have to be executed inthe order presented, or in any particular order, unless otherwiseindicated. Moreover, various activities described with respect to thelogic flows can be executed in serial or parallel fashion. The logicflows may be implemented using one or more hardware elements and/orsoftware elements of the described embodiments or alternative elementsas desired for a given set of design and performance constraints. Forexample, the logic flows may be implemented as logic (e.g., computerprogram instructions) for execution by a logic device (e.g., ageneral-purpose or specific-purpose computer).

FIG. 8 illustrates a flowchart 800 of a method for managing dataelements in a lockless FIFO/LIFO queue, according to exampleembodiments.

In a step 805, data elements having individual priority types forplacement in a queue are received. In a step 810, the priority types ofthe received data elements are utilized to determine placement in thequeue relative to an initial location established when a first dataelement is placed in an empty queue in order to manage the queue with acombination of first-in first-out and last-in first-out queuefunctionality.

FIG. 9 illustrates a flowchart 900 of a method for removing dataelements from a lockless FIFO/LIFO queue, according to exampleembodiments.

In a step 905, additional data elements are prevented from being addedto a linked list including one or more data elements. In a step 910, anoldest data element relative to the existing data elements in the queueis identified, the identified oldest data element being at a head of thelinked list. In a step 915, the linked list is removed from the queue.

VI. Example Computer Implementation Embodiment

In an example embodiment, the systems and methods described herein areimplemented using one or more well-known computers, such as computer 700shown in FIG. 7.

Computer 1000 can be any commercially available and well known computercapable of performing the functions described herein, such as computersavailable from International Business Machines, Apple, Sun, HP, Dell,Compaq, Digital, Cray, etc.

Computer 1000 includes one or more processors (also called centralprocessing units, or CPUs), such as a processor 1006. The processor 1006is connected to a communication bus 1004. Processors 1006 may includeany conventional or special purpose processor, including, but notlimited to, digital signal processor (DSP), field programmable gatearray (FPGA), and application specific integrated circuit (ASIC).

Computer 1000 includes one or more graphics processing units (alsocalled GPUs), such as GPU 1007. GPU 1007 is a specialized processor thatexecutes instructions and programs selected for complex graphics andmathematical operations in parallel.

Computer 1000 also includes a main or primary memory 1008, such asrandom access memory (RAM). The primary memory 1008 has stored thereincontrol logic 1028A (computer software), and data.

Computer 1000 also includes one or more secondary storage devices 1010.The secondary storage devices 1010 include, for example, a hard diskdrive 1012 and/or a removable storage device or drive 1014, as well asother types of storage devices, such as memory cards and memory sticks.The removable storage drive 1014 represents a floppy disk drive, amagnetic tape drive, a compact disk drive, an optical storage device,tape backup, etc.

The removable storage drive 1014 interacts with a removable storage unit1016. The removable storage unit 1016 includes a computer useable orreadable storage medium 1024 having stored therein computer software1028B (control logic) and/or data. Removable storage unit 1016represents a floppy disk, magnetic tape, compact disk, DVD, opticalstorage disk, or any other computer data storage device. The removablestorage drive 1014 reads from and/or writes to the removable storageunit 1016 in a well-known manner.

Computer 1000 also includes input/output/display devices 1022, such asmonitors, keyboards, pointing devices, touch-screen displays, etc.

Computer 1000 further includes a communication or network interface1018. The network interface 1018 enables the computer 1000 tocommunicate with remote devices. For example, the network interface 1018allows computer 1000 to communicate over communication networks ormediums 1024B (representing a form of a computer useable or readablemedium), such as LANs, WANs, the Internet, etc. The network interface1018 may interface with remote sites or networks via wired or wirelessconnections.

Control logic 1028C may be transmitted to and from computer 1000 via thecommunication medium 1024B. More particularly, the computer 1000 mayreceive and transmit carrier waves (electromagnetic signals) modulatedwith control logic 1030 via the communication medium 1024B.

Any apparatus or manufacture comprising a computer useable or readablemedium having control logic (software) stored therein is referred toherein as a computer program product or program storage device. Thisincludes, but is not limited to, the computer 1000, the main memory1008, the secondary storage devices 1010, the removable storage unit1016 and the carrier waves modulated with control logic 1030. Suchcomputer program products, having control logic stored therein that,when executed by one or more data processing devices, cause such dataprocessing devices to operate as described herein, represent embodimentsof the invention.

Embodiments can work with software, hardware, and/or operating systemimplementations other than those described herein. Any software,hardware, and operating system implementations suitable for performingthe functions described herein can be used.

It is to be appreciated that the Detailed Description section, and notthe Summary and Abstract sections, is intended to be used to interpretthe claims. The Summary and Abstract sections may set forth one or more,but not all, exemplary embodiments of the invention as contemplated bythe inventor(s), and thus, are not intended to limit the invention andthe appended claims in any way.

The embodiments have been described above with the aid of functionalbuilding blocks illustrating the implementation of specified functionsand relationships thereof. The boundaries of these functional buildingblocks have been arbitrarily defined herein for the convenience of thedescription. Alternate boundaries can be defined so long as thespecified functions and relationships thereof are appropriatelyperformed.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the embodiments that others can, byapplying knowledge within the skill of the art, readily modify and/oradapt for various applications such specific embodiments, without undueexperimentation, without departing from the general concept of theinvention. Therefore, such adaptations and modifications are intended tobe within the meaning and range of equivalents of the disclosedembodiments, based on the teaching and guidance presented herein. It isto be understood that the phraseology or terminology herein is for thepurpose of description and not of limitation, such that the terminologyor phraseology of the specification is to be interpreted by the skilledartisan in light of the teachings and guidance.

The breadth and scope of the invention should not be limited by any ofthe above-described exemplary embodiments, but should be defined only inaccordance with the following claims and their equivalents.

What is claimed is:
 1. A method for managing a lockless queuecomprising: receiving data elements having individual priority types forplacement in a queue; and utilizing the priority types of the receiveddata elements to determine placement in the queue relative to an initiallocation established when a first data element is placed in an emptyqueue in order to manage the queue with a combination of first-infirst-out and last-in first-out queue functionality.
 2. The method ofclaim 1, further comprising: determining whether the queue is empty;when the queue is determined to be empty, inserting a received dataelement at the initial location in the queue; and when the queue isdetermined to be not empty, determining whether the received dataelement is of a first priority type or a second priority type, whereinthe utilizing the priority types includes: when the received dataelement is determined to be of the first priority type, inserting thereceived data element in the queue after each existing data element ofthe first priority type in the queue; and when the received data elementis determined to be of the second priority type, inserting the receiveddata element in the queue before each existing data element of the firstpriority type after the initial location in the queue.
 3. The method ofclaim 1, wherein the utilizing the priority types of the received dataelements to determine placement in the queue includes inserting thereceived data elements into a linked list.
 4. The method of claim 2,wherein the data element at the initial location in the queue is anoldest data element relative to the existing data elements in the queue.5. The method of claim 2, wherein the inserting the received dataelement in the queue before each existing data element of the firstpriority type after the initial location in the queue includes:identifying a most recently inserted data element in a linked list, themost recently inserted data element including a reference, and thelinked list including an oldest data element in the queue; and insertingthe received data element of the second priority type into the linkedlist by updating the reference included in the identified most recentlyinserted data element to reference the received data element of thesecond priority type.
 6. The method of claim 1, further comprisingremoving one or more data elements from the queue, wherein the removingone or more data elements includes removing a linked list from thequeue, the linked list including an oldest data element in the queue. 7.The method of claim 1, further comprising removing one or more dataelements from the queue, wherein the removing one or more data elementsincludes: preventing additional data elements from being added to alinked list including one or more data elements; identifying an oldestdata element relative to the existing data elements in the queue, theidentified oldest data element being at a head of the linked list; andremoving the linked list from the queue.
 8. The method of claim 7,further comprising: traversing the removed linked list starting at thehead of the removed linked list; and processing one or more dataelements in the removed linked list based on an order in which theremoved linked list is traversed.
 9. The method of claim 7, wherein thepreventing additional data elements from being added to a linked listincludes: identifying a most recently inserted data element in thelinked list, the most recently inserted data element including areference to a value; and updating the reference included in the mostrecently inserted data element to reference a stop node.
 10. The methodof claim 9, further including: identifying a list buffer including areference; updating the reference included in the list buffer toreference the data element at the initial location in the queue.
 11. Themethod of claim 9, further including: after the preventing additionaldata elements from being added to a linked list, inserting a secondreceived data element in the queue, the inserting a second received dataelement in the queue including: determining whether the queue is empty;when the queue is determined to be empty, inserting the second receiveddata element at a second location, after the initial location, in thequeue; when the queue is determined to be not empty, determining whetherthe second received data element is of a first priority type or a secondpriority type; when the second received data element is determined to beof the first priority type, inserting the second received data elementin the queue after each existing data element of the first priority typein the queue; and when the second received data element is determined tobe of the second priority type, inserting the second received dataelement in a second linked list, different from the first linked list,in the queue.
 12. A system for maintaining a lockless queue comprising:at least one processing device configured to process: a queue inserterconfigured to: receive data elements having individual priority typesfor placement in a queue; and utilize the priority types of the receiveddata elements to determine placement in the queue relative to an initiallocation established when a first data element is placed in an emptyqueue in order to manage the queue with a combination of first-infirst-out and last-in first-out queue functionality.
 13. The system ofclaim 12, wherein the queue inserter is configured to: a queue inserterconfigured to: determine whether the queue is empty; when the queue isdetermined to be empty, insert a received data element at the initiallocation in the queue; when the queue is determined to be not empty,determine whether the received data element is of a first priority typeor a second priority type; when the received data element is determinedto be of the first priority type, insert the received data element inthe queue after each existing data element of the first priority type inthe queue; and when the received data element is determined to be of thesecond priority type, insert the received data element in the queuebefore each existing data element of the first priority type after theinitial location in the queue.
 14. The system of claim 13, wherein whenthe queue inserter inserts the received data element in the queue beforeeach existing data element of the first priority type after the initiallocation in the queue, the queue inserter inserts the received dataelement into a linked list.
 15. The system of claim 13, wherein the dataelement at the initial location in the queue is an oldest data elementrelative to the existing data elements in the queue.
 16. The system ofclaim 13, wherein when the queue inserter inserts the received dataelement in the queue before each existing data element of the firstpriority type after the initial location in the queue, the queueinserter is configured to: identify a most recently inserted dataelement in a linked list, wherein the most recently inserted dataelement includes a reference to a value, and the linked list includes anoldest data element in the queue; and insert the received data elementof the second priority type into the linked list by updating thereference included in the identified most recently inserted data elementto reference the received data element of the second priority type. 17.The system of claim 12, further comprising a queue remover configured toremove one or more data elements, wherein when the queue remover removesone or more data elements, the queue remover is configured to: remove alinked list from the queue, wherein the linked list includes an oldestdata element in the queue.
 18. The system of claim 12, furthercomprising a queue remover configured to remove one or more dataelements, wherein when the queue remover removes one or more dataelements, the queue remover is configured to: prevent additional dataelements from being added to a linked list including one or more dataelements; identify an oldest data element relative to the existing dataelements in the queue, the identified oldest data element being at ahead of the linked list; remove the linked list from the queue; traversethe removed linked list starting at the head of the linked list; andprocess one or more data elements in the removed linked list based on anorder in which the removed linked list is traversed.
 19. The system ofclaim 18, wherein when the queue remover prevents additional dataelements from being added to a linked list, the queue remover isconfigured to: identify a most recently inserted data element in thelinked list, wherein the most recently inserted data element includes areference to a value; and update the reference included in the mostrecently inserted data element to reference a stop node.
 20. Acomputer-readable medium having instructions stored thereon, theinstructions causing at least one computing device to performoperations, the operations comprising: receiving data elements havingindividual priority types for placement in a queue; and utilizing thepriority types of the received data elements to determine placement inthe queue relative to an initial location established when a first dataelement is placed in an empty queue in order to manage the queue with acombination of first-in first-out and last-in first-out queuefunctionality.